Using Explainable Artificial Intelligence to Predict Potentially Preventable Hospitalizations: A Population-Based Cohort Study in Denmark.

BACKGROUND: The increasing aging population and limited health care resources have placed new demands on the healthcare sector. Reducing the number of hospitalizations has become a political priority in many countries, and special focus has been directed at potentially preventable hospitalizations. OBJECTIVES: We aimed to develop an artificial intelligence (AI) prediction model for potentially preventable hospitalizations in the coming year, and to apply explainable AI to identify predictors of hospitalization and their interaction. METHODS: We used the Danish CROSS-TRACKS cohort and included citizens in 2016-2017. We predicted potentially preventable hospitalizations within the following year using the citizens' sociodemographic characteristics, clinical characteristics, and health care utilization as predictors. Extreme gradient boosting was used to predict potentially preventable hospitalizations with Shapley additive explanations values serving to explain the impact of each predictor. We reported the area under the receiver operating characteristic curve, the area under the precision-recall curve, and 95% confidence intervals (CI) based on five-fold cross-validation. RESULTS: The best performing prediction model showed an area under the receiver operating characteristic curve of 0.789 (CI: 0.782-0.795) and an area under the precision-recall curve of 0.232 (CI: 0.219-0.246). The predictors with the highest impact on the prediction model were age, prescription drugs for obstructive airway diseases, antibiotics, and use of municipality services. We found an interaction between age and use of municipality services, suggesting that citizens aged 75+ years receiving municipality services had a lower risk of potentially preventable hospitalization. CONCLUSION: AI is suitable for predicting potentially preventable hospitalizations. The municipality-based health services seem to have a preventive effect on potentially preventable hospitalizations.

The Framing of machine learning risk prediction models illustrated by evaluation of sepsis in general wards.

Problem framing is critical to developing risk prediction models because all subsequent development work and evaluation takes place within the context of how a problem has been framed and explicit documentation of framing choices makes it easier to compare evaluation metrics between published studies. In this work, we introduce the basic concepts of framing, including prediction windows, observation windows, window shifts and event-triggers for a prediction that strongly affects the risk of clinician fatigue caused by false positives. Building on this, we apply four different framing structures to the same generic dataset, using a sepsis risk prediction model as an example, and evaluate how framing affects model performance and learning. Our results show that an apparently good model with strong evaluation results in both discrimination and calibration is not necessarily clinically usable. Therefore, it is important to assess the results of objective evaluations within the context of more subjective evaluations of how a model is framed.

Cohort profile: CROSS-TRACKS: a population-based open cohort across healthcare sectors in Denmark.

PURPOSE: This paper describes the open cohort CROSS-TRACKS, which comprises population-based data from primary care, secondary care and national registries to study patient pathways and transitions across sectors while adjusting for sociodemographic characteristics. PARTICIPANTS: A total of 221 283 individuals resided in the four Danish municipalities that constituted the catchment area of Horsens Regional Hospital in 2012-2018. A total of 96% of the population used primary care, 35% received at least one transfer payment and 66% was in contact with a hospital at least once in the period. Additional clinical information is available for hospital contacts (eg, alcohol intake, smoking status, body mass index and blood pressure). A total of 23% (n=8191) of individuals aged ≥65 years had at least one potentially preventable hospital admission, and 73% (n=5941) of these individuals had more than one. FINDINGS TO DATE: The cohort is currently used for research projects in epidemiology and artificial intelligence. These projects comprise a prediction model for potentially preventable hospital admissions, a clinical decision support system based on artificial intelligence, prevention of medication errors in the transition between sectors, health behaviour and sociodemographic characteristics of men and women prior to fertility treatment, and a recently published study applying machine learning methods for early detection of sepsis. FUTURE PLANS: The CROSS-TRACKS cohort will be expanded to comprise the entire Central Denmark Region consisting of 1.3 million residents. The cohort can provide new knowledge on how to best organise interventions across healthcare sectors and prevent potentially preventable hospital admissions. Such knowledge would benefit both the individual citizen and society as a whole.

Explainable artificial intelligence model to predict acute critical illness from electronic health records.

Acute critical illness is often preceded by deterioration of routinely measured clinical parameters, e.g., blood pressure and heart rate. Early clinical prediction is typically based on manually calculated screening metrics that simply weigh these parameters, such as early warning scores (EWS). The predictive performance of EWSs yields a tradeoff between sensitivity and specificity that can lead to negative outcomes for the patient. Previous work on electronic health records (EHR) trained artificial intelligence (AI) systems offers promising results with high levels of predictive performance in relation to the early, real-time prediction of acute critical illness. However, without insight into the complex decisions by such system, clinical translation is hindered. Here, we present an explainable AI early warning score (xAI-EWS) system for early detection of acute critical illness. xAI-EWS potentiates clinical translation by accompanying a prediction with information on the EHR data explaining it.

Early detection of sepsis utilizing deep learning on electronic health record event sequences.

BACKGROUND: The timeliness of detection of a sepsis incidence in progress is a crucial factor in the outcome for the patient. Machine learning models built from data in electronic health records can be used as an effective tool for improving this timeliness, but so far, the potential for clinical implementations has been largely limited to studies in intensive care units. This study will employ a richer data set that will expand the applicability of these models beyond intensive care units. Furthermore, we will circumvent several important limitations that have been found in the literature: (1) Model evaluations neglect the clinical consequences of a decision to start, or not start, an intervention for sepsis. (2) Models are evaluated shortly before sepsis onset without considering interventions already initiated. (3) Machine learning models are built on a restricted set of clinical parameters, which are not necessarily measured in all departments. (4) Model performance is limited by current knowledge of sepsis, as feature interactions and time dependencies are hard-coded into the model. METHODS: In this study, we present a model to overcome these shortcomings using a deep learning approach on a diverse multicenter data set. We used retrospective data from multiple Danish hospitals over a seven-year period. Our sepsis detection system is constructed as a combination of a convolutional neural network and a long short-term memory network. We assess model quality by standard concepts of accuracy as well as clinical usefulness, and we suggest a retrospective assessment of interventions by looking at intravenous antibiotics and blood cultures preceding the prediction time. RESULTS: Results show performance ranging from AUROC 0.856 (3 h before sepsis onset) to AUROC 0.756 (24 h before sepsis onset). Evaluating the clinical utility of the model, we find that a large proportion of septic patients did not receive antibiotic treatment or blood culture at the time of the sepsis prediction, and the model could, therefore, facilitate such interventions at an earlier point in time. CONCLUSION: We present a deep learning system for early detection of sepsis that can learn characteristics of the key factors and interactions from the raw event sequence data itself, without relying on a labor-intensive feature extraction work. Our system outperforms baseline models, such as gradient boosting, which rely on specific data elements and therefore suffer from many missing values in our dataset.